Performance Evaluation of Virtual Appliances

Virtualization technology has become standard in many computing environments. Independent Software Vendors (ISVs) are now using virtual machines to deliver multiple software appliances. However, the performance of these appliances when run on virtualized platforms is not well understood. In this paper, we present a performance evaluation framework for virtualized appliances. We are using the XenServer virtual machine system and evaluate its performance on four different classes of workload: 1) voice-over-IP, 2) TCPIP networking, 3) IO disk intensive, and 4) CPU intensive. Our primary interest is understanding the scalability of the VoIP application in the presence of the other three classes of workload. We also consider a range of different virtual machine configurations. Our experiments show that the XenSource VMM can isolate hardware resource utilization (i.e., disk and network I/O) nicely across multiple virtual machines. But when running multiple virtual appliances concurrently, sharing physical CPUs across virtual machines introduces significant overhead. We present the results of this study and consider future work to address some of these performance issues.